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We study the stability and information encoding capacity of synchronized states in a neuronal network 
model that represents part of thalamic circuitry. Our model neurons have a Hodgkin-Huxley-type low threshold 
Calcium channel, display post inhibitory rebound, and are connected via GABAergic inhibitory synapses. 

We find that there is a threshold in synaptic strength, Tc, below which there are no stable spiking network 
states. Above threshold the stable spiking state is a cluster state, where different groups of neurons fire consec- 
utively, and each neuron fires with the same cluster each time. Weak noise destabilizes this state, but stronger 
noise drives the system into a different, self-organized, stochastically synchronized state. Neuronal firing is still 
organized in clusters, but individual neurons can hop from cluster to cluster Noise can actually induce and 
sustain such a state below the threshold of synaptic strength. We do find a qualitative difference in the firing 
patterns between small (~ 10 neurons) and large 1000 neurons) networks. 

We determine the information content of the spike trains in terms of two separate contributions: the spike 
time jitter around cluster firing times, and the hopping from cluster to cluster We quantify the information loss 
due to temporally correlated interspike intervals. Recent experiments on the locust olfactory system and striatal 
neurons suggest that the nervous system may actually use these two channels to encode separate and unique 
information. 
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I. INTRODUCTION 

The brain receives an enormous amount of information 
transduced by peripheral sense organs. This massive infor- 
mation influx is coded and decoded in ways that are not yet 
fully understood in cognitive neuroscience. Recent studies 
have focussed on specific neural substrates for binding mech- 
anisms. Binding is the process by which the brain combines 
different aspects of sensory modalities of one object into one 
unified percept. The neural mechanisms that underlie syn- 
chronization in different parts of the brain are only partly un- 
derstood. There is the suggestion that synchronization may be 
relevant to binding ^ . Recent experiments have shown that 
inhibitory interneurons in the hippocampus |^], the thalamic 
reticular nucleus and the locust olfactory system [Q| can 
indeed synchronize neuronal discharges. Subsequent theoreti- 
cal analysis of networks of interneurons has shown that strong 
synchronization by mutual inhibition is only moderately ro- 
bust against neuronal heterogeneities [^] and synaptic noise 
1^]. In strong synchronization all the neurons fire with a short 
time-interval from each other. 

In most experiments to date one measures the activity of 
one neuron, or a small population of neurons. Periodic os- 
cillations (extracellular, or subthreshold intracellular) mea- 
sured in these experiments are consistent with strong as well 
as weak synchronization. In weak synchronization the aver- 
age neuronal activity is periodic, without each individual neu- 
ron having to fire at each period. Often theoretical analyses, 
however, have focussed on strong synchronization. Here we 
conjecture that weak synchronization is robust against neu- 
ronal heterogeneities and synaptic noise, and consequently it 
is much more likely to occur in neuronal systems. Further- 
more we show that it can encode more information compared 
to strongly synchronized states. We present numerical results 
of weak synchronization in a simple model of a network of 
Thalamic neurons that supports our conjecture. We use a tha- 
lamic network, as an example, due to the wealth of modeling 
information that is already available. The mechanism we dis- 
cuss here, however, has more general applicability. 

The thalamus acts as a relay for most of the sensory infor- 
mation that travels to cortical structures. It regulates sleep- 
wake cycles ||^] and it may be involved in early stimulus bind- 
ing [^. The lateral geniculate nucleus (LGN) and thalamic 
reticular nucleus (TRN) that are involved in vision have been 
studied extensively. Neurons of the thalamus express low 
threshold Calcium currents [^, and they rebound after a sus- 
tained hyperpolarization. It has been shown experimentally 
and in model calculations that inhibitory neurons can synchro- 
nize neuronal disch^'ges in the thalamus [p[-pj|] and produce 
traveling waves [p^j-p^. 

The hallmark of weak synchronization is multimodal in- 
terspike interval (ISI) histograms (ISIH). The ISI occurs only 
near multiples of a particular time-scale, e.g. the period of the 
population activity T. Multimodality of the ISIH has been 
observed in the LGN [|l8]], and it was attributed to the ac- 
tion of inhibitory neurons. Multimodal ISIH have also been 
found in model simulations of coupled inhibitory networks in 



the presence of noise [ |19| ] and in systems exhibiting stochas- 
tic resonance (SR) due to a periodic drive |20|. A theoretical 
mechanism for autonomous stochastic resonance (ASR) was 
proposed in a recent paper [^T|]. There the periodic drive was 
replaced by a periodic mode in an internal kinetic variable, 
such that the spikes ride on top of subthreshold voltage oscil- 
lations. In our work the periodic neuronal activity in the noise- 
driven system is self-induced by the network. This mechanism 
is absent in unconnected single neurons, or in a single element 
with autosynaptic feedback. 

It has been suggested that the brain may encode informa- 
tion through an ensemble or cluster of neurons that fire within 
a short time of each other [ p2[p3| ]. A particular neuron may be 
part of a cluster for a few cycles, before it joins another neu- 
ronal ensemble. This type of dynamics is very similar to the 
neuronal clusters that form in our model simulations described 
below. An important problem is how to quantify the informa- 
tion content of these binding-like cluster states. The Shannon 
entropy has been used as a measure of information content 
in investigations of sensory neurons in, for instance, crickets 
p3], and flies |25|. It is, nonetheless, not known how the 



brain processes information, and thus it is not clear whether 
the Shannon entropy is the correct quantity for this purpose. 
It does, however, provide an upper bound on the theoretical 
information content of spiking neurons. It also implies that 
noise in the nervous system contains information, and that 
noisy neurons are transmitting more information compared 
to regular noiseless spiking neurons. We emphasize that this 
statement is still controversial [|2^-p^, because even if the en- 
tropy measure yields consistent results in sensory systems this 
does not guarantee its relevance to the central nervous system. 
With these caveats in mind we still proceed to characterize the 
information content of our neural networks by calculating its 
well-defined Shannon entropy. 

One would like to calculate both the output entropy of the 
model system and the mutual information. The mutual in- 
formation quantifies how the ensemble of outputs is related to 
one of the possible realizations of the input, and it involves ad- 
ditional averagings over a conditional probability distribution 
which makes it very hard to calculate. Even the simpler cal- 
culation of the Shannon entropy from its definition in terms of 
the spike times is a difficult calculation. Here we shall focus 
on the Shannon entropy of the neuronal output of a neuron as 
part of the complete network. We also present here some ap- 
proximations that allows us to estimate the Shannon entropy 
using the interspike interval time series. 



II. METHODS 

A. Network model 

Our single neuron model equation contains a low threshold 
Calcium current Ica, a general leak current II, a synaptic 
current I syn, and a noise current Cm£,, 
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together with the first order (Hodgkin-Huxley type) kinetic 
equations for the activation m and inactivation h variables 
for Ica and the synaptic variable s. This yields a neuronal 
dynamics in terms of four variables, V, m, h, and s. We 
have used the kinetics for Ica and Isyn as specified in [ p^ 
(a detailed description of the model is given in Appendix A). 
Our single neuron model captures some important features 
of the dynamics of thalamic neurons, in particular its post 
inhibitory rebound (PIR). We are presently studying a more 
complete model, incorporating thalamo-cortical relay neurons 
and GABAergic thalamic reticular neurons [p3|], including all 
the relevant active currents | ]2^ , p0[ |. Our preliminary results 
suggest that this does not change the conclusions of our dis- 
cussion here. 

The neurons in our network are connected all-to-all by in- 
hibitory GABAergic synapses. Previous studies [ p^p^ have 
shown that the precise spatial connectivity is important for the 
activity propagation. In this work, we will not consider the 
spatial characteristics of the neuronal activity. We have stud- 
ied different sized systems, varying from iV = 1 (a single 
neuron with autosynaptic feedback) to = 1000. We also 
have included two types of noises in our model, either Gaus- 
sian current noise, characterized by {£,) = and 



^syn 



^^^^ 



(3) 



mm) = 2D6{t), 
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with D the strength of the noise, or with Poisson distributed 
excitatory post-synaptic potentials (EPSPs) and inhibitory 
post-synaptic potentials (IPSPs). In our previous work we 
have shown that these two types of noises are not fully equiv- 
alent |31 1. Both can generate, however, similar statistics, and 



theoretically Gaussian noise is easier to control and vary. The 
results presented here are thus obtained with Gaussian noise. 
Unless stated differently the physiological total synaptic con- 
ductance used is = 2 mS/cm^, and the decay time of the 
synaptic channel Ts — 16 ms. The noise strength D is ex- 
pressed in units of mV^ /ms, time in ms, currents in fiA / cm? , 
and voltage in mV. 

The resulting differential equations for Vi, mi, hi, Si are 
numerically integrated using a noise-adapted second order 
Runge-Kutta algorithm [^] with a time-step rfi = 0.1 ms. The 
calculation starts with random initial conditions, with the ini- 
tial voltage chosen from a uniform distribution with a range 
of 20 mV centered around —68 mV, and m, h, and s are set 
equal to their asymptotic values for a given value of V . 



B. Calculated quantities 

The raw model output are the time-traces for Vi, mi, hi, and 
Si. The spike-times are defined as the time when the voltage 
Vi crosses —30 mV from below. We determined the standard 
histograms of interspike intervals p2[ ] . The instantaneous fir- 
ing rate, or frequency f, is defined as the number of action 
potentials per second in a bin of 2 ms. Both the ISI histogram 
and / are averaged over all neurons in the network. We also 
calculated 



which is proportional to the current drive due to the synaptic 
connections with other neurons (and itself). Because the net- 
work is connected all-to-all, Vsyn is the same for each neuron 
and represents an averaged or mean field type drive. The vari- 
able h determines the excitability of the neuron, and the state 
of the network strongly depends on the /i-value distribution. 
When Vsyn is below a certain value the network is disinhib- 
ited and will fire shortly. To determine the /i-distribution prior 
to firing we have chosen v^yn = 0.01 as the threshold. This 
value is reached every cycle, except in the presence of strong 
noise. In that case the disperse nature of the firing creates an 
average value of Vgyn above 0.01. We have determined both 
the instantaneous as well as the time-averaged distribution of 
h. 



III. RESULTS 

The model considered here contains an inward low thresh- 
old Calcium current. It, that initiates the Calcium action po- 
tentials. It is inactivated at the resting membrane potential 
(RMP, equal to —65.57 mV), and it is de-inactivated at hyper- 
polarized voltages. For the neuron to be excitable, h has to be 
de-inactivated (i.e. h > 0.305). We illustrate this in Fig.^a). 
There are two V nuU-clines drawn, one (I) in the absence of a 
current, and the other (II) in the presence of a constant hyper- 
polarizing current i = —1 jiA/cm^. We apply a short (10ms) 
and a long pulse (200ms) with strength i. The phase point 
moves to II, and the h value starts increasing with time-scale 
Ti = 500 ms (see Appendix A). Upon termination of the short 
pulse the phase point moves back to I, without generating an 
action potential (AP). When the long pulse ends, however, the 
/i-value is too high, and the phase point misses the nearby 
branch of I, and generates an AP. 

The necessary hyperpolarization is supplied by the in- 
hibitory postsynaptic potential (IPSP) generated by the activ- 
ity of other neurons in the network. The strength of the IPSPs 
is determined by the value of the synaptic conductance gs and 
the decay time r^. The critical value for periodic oscillations 
is defined as Tg — Tc{gs, N), that depends on the number N 
of neurons in the network. For Tg < Tc the oscillation dies 
out after a finite number of action potentials. In Fig. |l](b) we 
show the voltage trace oscillations below and above threshold 
for = 1 (single neuron with autosynaptic feedback). At 
the start of the simulation the neuron is released from a hyper- 
polarized voltage. For subthreshold values of Tg the neuron 
produces a few spikes before returning to RMP. During each 
spike the average h value decreases, since the inhibitory drive 
is not strong enough to replenish the loss due to the depolar- 
ization of the AP. Above threshold a periodic spike train is 
produced. The /i-value varies periodically, it decreases dur- 
ing the AP and it increases during the inhibition. Note that 
the interspike intervals are determined by Tg. The subthresh- 
old spike train has therefore a much smaller ISI compared 
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to the one above threshold. We have determined the bound- 
ary between stable and unstable oscillations as function of gs 
(Fig. |l](a)). The minimum duration of the IPSP needed 
for deinactivation, increases for weaker synaptic coupling gs- 
The situation for a real network {N > 1) is more compli- 
cated, since the initial voltages play an important role. If, for 
instance, we would start with neurons clamped at their resting 
membrane potential nothing will happen. To obtain a spiking 
network state we therefore always start the simulations with 
part or all the neurons clamped at hyperpolarizing voltages. 
With uniform initial conditions all neurons are clamped at the 
same voltage value. The threshold Tc for self-sustained oscil- 
lations is then equal to the one for a single neuron (Fig.^(a)). 
Above threshold this network is in a coherent state: all neu- 
rons spike at the same time. For random initial conditions 
the initial voltage is chosen from a uniform distribution with 
a range of 20 mV around a hyperpolarized average value. In 
that case the network can sustain stable oscillations for lower 
values of (Fig.||(b)). The network settles in a state where 
groups of neurons fire simultaneously. It is easy to understand 
why such cluster states emerge. Starting from random initial 
conditions each neuron will have a different phase, and will 
thus reach the AP threshold at a different time. The first neu- 
rons to fire will cause an inhibition that blocks other neurons 
(further from threshold) from firing. They can only fire after 
the decay of the inhibition (a few r^). Periodic oscillations 
in the network are sustained by inhibition waves produced by 
the activity of de-inactivated neurons. The oscillations auto- 
matically become coherent, with the initial phase differences 
between cluster neurons driven to zero. In the simplest cluster 
state each neuron fires with the same ISI. The time between 
consecutive cluster firings, or cycle length, may vary since the 
strength of inhibition depends on the cluster size. Neurons 
will only fire when Vgyn is below a certain value: the higher 
the initial value (proportional to cluster size), the longer it 
takes to reach this value. More complex cluster states may 
also contain neurons that fire with different frequencies. 

Sufficiently strong noise can induce and maintain a spiking 
network state even for < Tc- Again we have to distinguish 
between single neurons and a network. The noise-induced dy- 
namics of a single neuron with autosynaptic feedback will not 
yield a periodic spike-trace. Instead, the ISI distribution has 
a peak for short times due to ISIs within the spike trains, and 
an exponential distribution for the intervals between the end 
of one, and the start of another spike train. We show some 
representative voltage traces in Fig. ^(c). Close to threshold 
and with weak noise the neuron produces a long transient that 
dies out eventually. Stronger noise can spontaneously induce 
a spike. The inhibition induced by the AP then manages to 
produce a short spike train. The number of spikes in this 
train depends on the distance from threshold. This makes the 
dynamics discussed here essentially different from Stochastic 
Resonance, since in that case one would obtain a multimodal 
distribution, with peaks at multiples of the driving frequency. 

We studied the dynamics of the cluster states in a network 
with N — 1000, in terms of the variables shown in Figs, ^and 
§ We focused on four values for D = 0, 0.0024, 0.008, and 
0.8. An important variable in our analysis is h, the inactiva- 



tion variable of It, since the model neuron is only excitable 
when h is large enough. The distribution of h values in the 
network will tell us which neurons are excitable, and which 
ones need to be de-inactivated by further inhibition cycles. 
The regularity of the network dynamics is further reflected in 
the periodicity of / and Vsyn (Fig. |]) and their autocorrela- 
tion function (not shown). Note that Vgyn itself acts as a drive 
on the neurons. The larger the distance between the peak and 
trough, the stronger the synchronizing force. 

We can identify four different types of regimes. For zero 
noise (D=0) the system is in a state with five clusters of un- 
equal size. The distribution of cluster sizes is determined by 
the initial conditions. At each time a neuron can only have 
one of five h-values. This set of h- values goes through a mod- 
ulation with a period of five cycles (Fig ^a)). The derived 
quantities V, h, Vsyn, and f (Fig. |](a)) go through the same 
modulations. When all the clusters have the same size, there 
are no such modulations, and the h histogram would consist 
of only 5 peaks. 

Immediately after firing, the neuron is partially de- 
inactivated with each subsequent cycle until it is excitable 
again (see the h time traces in Fig. |]). The neuron then has 
to wait its turn to become disinhibited and fire before other 
clusters do. When there is noise, there is dispersion in the 
spike firing times. In the absence of time delay there is only 
a short time interval during which neurons can fire before the 
inhibition blocks all other firings during one cycle. As a re- 
sult, clusters lose neurons whose AP has been noise delayed, 
and other clusters gain those neurons as members. In addition, 
noise can cause neurons to fire before the rest in their cluster 

Weak noise (D=0.0024) disorders the system. The network 
starts out with unequal cluster sizes (due to the initial condi- 
tions). Large clusters lose more neuron members than smaller 
ones. Weak noise, however, is not strong enough to equalize 
their numbers on the time-scale considered (5 x 10^ ms). In- 
stead the cluster sizes start to vary in a somewhat stochastic 
fashion, leading to an erratic firing rate (Fig. 0(b)) and a fluc- 
tuating period. The time-averaged h histogram is very broad 
(Fig. |b)). 

For stronger noise (D=0.008) the average cluster size be- 
comes stationary after a brief transient. The /i-values that the 
neurons of different clusters cycle go through are, on the aver- 
age, the same. As a result there are six smooth peaks in the h- 
histogram. The peaks become sharper for more de-inactivated 
values, and the h-distribution is stationary (on the average it is 
the same for each cycle). The actual neurons that fire in each 
cluster changes with time. This state is stable up to a noise 
strength of approximately D=l (for N=1000). 

The cycle-to-cycle fluctuations in cluster sizes increases 
with increasing D. The inhibition that each cluster receives 
(proportional to Vsyn) varies, and as a result the width of the 
peaks in the h histogram increases. The cluster size also de- 
creases with increasing D, and the amplitude of Vgyn oscilla- 
tions also decreases. The stability of a cluster can be defined 
as the fraction of neurons that are still part of a given clus- 
ter the next time it fires. This is related to the number and 
height of the peaks in the ISIH (see Fig. ^a) and (b)), and it 
decreases with D. The neuron spends most of its time in an 
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excited state (flat part of h{t). Fig. 5(d)) waiting for its noise 
induced AP threshold to fall in the inhibition free window. 
For that reason it is unlikely to fire with the same cluster as in 
the previous time. Finally, for larger noise strengths, D > 1, 
the firing is no longer organized in clusters, since the noise 
has become so large that during an inhibition free window not 
enough neurons fire coherently to create a large enough inhi- 
bition to block the discharge until the next cycle. As a result 
there are no quiescent periods defining cycles and no distinct 
cycles either 

The cluster states can be quantitatively described by the av- 
erage cycle length (period), and the periodicity (number of cy- 
cles) of the response. We have studied these two quantities as 
a function of D for different values of Tg for N=1000, and for 
different system sizes for Ts = 16. For smaller Ts the amount 
of inhibition available for de-inactivation is lower, and the pe- 
riodicity increases since the de-inactivation has to be spread 
out over more cycles. The cycle-period, being proportional to 
Tg, also decreases. In addition, the oscillation needs a higher 
minimum value of D to sustain itself, since the average clus- 
ter size is given by the system size over the periodicity. When 
it becomes too small, the periodic component in the inhibi- 
tion becomes too small, and as a result the cluster state dies. 
The noise strength for which this happens is only weakly de- 
pendent on Ts- Note that the periodic component is propor- 
tional to the cluster size normalized by the network size, in 
addition it decreases with increasing jitter. There is, however, 
a difference between large networks (N ^ 1000) and small 
ones (N ~ 10). For small networks, a single neuron can pro- 
vide enough inhibition to block neuronal discharge and thus 
to maintain a periodic network state. The state is very robust 
against noise, even when noise reduces the cluster to its min- 
imum size (one neuron), it can still maintain a spiking state. 
The drawback is that the fluctuations in cluster size will be of 
the order of the cluster size itself, causing the cycle length to 
vary considerably. Below threshold (not shown) the amount of 
noise needed to induce a spiking state increases, and for very 
small networks (N < 4) failure can occur, i.e. the network 
becomes quiescent if one neuron fails to fire. Note that the 
maximum periodicity that the system can sustain is bounded 
by the system size. In Fig. ^ we see that the periodicity ob- 
tained for a given D decreases with N. 

We now discuss the possible information content of the ISI 
time series of individual neurons. We assume that the states 
are characterized by a periodic population activity. This is re- 
flected in the ISI time series because the ISI will only take 
values close to multiples of the cycle length. The ISIH will 
therefore consist of a series of peaks. If a neuron would con- 
sistently spike with the same cluster, there would be only one 
peak. The peak with maximum weight is close to the average 
ISI (and periodicity) of the network, the other peaks corre- 
spond to the ISI where the neuron changed cluster The rel- 
ative weight of the maximum peak is thus a measure of the 
stability of the clusters. In Figs. 0(a), and ^(b) we show the 
ISIH of a state with stable and unstable clusters, respectively. 
The width of each peak represents the jitter around a multiple 
of T. We find that the ISIH of an individual network neuron 
is to a very good approximation the same as the population 



averaged ISIH. This does not mean that all the neurons fire 
independently, it only states that all individual neurons have 
identical properties, and that in the long run the statistics of 
their time-series are the same. Our analysis is therefore per- 
formed on the population averaged ISIH, and we also use the 
population averaged return map. We find that the ISIH is well 
described by a sum of Gaussians (SOG) of different widths 
(see Appendix B), and that the width increases with the the 
peak number. In Appendix B we derive an expression given 
in Eq. ( p^ for the entropy of the SOG. We see that the number 
of peaks and their weight -cluster hopping- and their width - 
jitter within the cluster- yield two distinct contributions to the 
information encoding capacity. In Fig. ^c) we plot the en- 
tropy as a function of width a (taken constant for all peaks) 
and the total number of n peaks (weighted with a cosine en- 
velope, Eq. (Bl 1). In our simulations we find that the amount 
of jitter and the number of peaks are closely correlated, and 
grow with D. 

Our present entropy calculations assume that there is no 
correlation between consecutive ISIs. For a given amount 
of correlation 7 (see Appendix B) the entropy per spike will 
decrease. We have quantified this in a model calculation for 
n — 2 and various amounts of correlation 7 (see Fig. ^d)). 



IV. DISCUSSION 

In recent years considerable attention, as well as contro- 
versy, has been directed at studying the variability of neuronal 
discharge in the cortex [Q. It is beyond doubt that neurons in 
vivo are noisy. The question is whether exact spike-times mat- 
ter - that is, if the jitter in spike times represents information, 
for instance quantified by the Shannon entropy - or if only 
the average firing rate matters. If spike times do matter, then 
the synchronized discharge has a special significance. An im- 
portant question is whether the nervous system is sensitive to 
synchronization or not. Our work is relevant in shedding light 
to this fundamental question in two ways. We have shown that 
noisy neurons can synchronize without the need of a strong 
external drive, and that the synchronized neuronal discharge 
has a potentially high information content. To place our re- 
sults in a proper context we will now discuss these points in 
more detail below. 

The role that inhibitory interneurons play in the function- 
ing of the nervous system has long been unclear It is hard 
to find, and to record from interneurons, because they are 
rather small. Moreover the output of the nervous system is 
mostly generated by the principal neurons, so early investiga- 
tions studied mainly the pyramidal neurons. In recent years 
it has become clear that inhibition plays a major role in syn- 
chronizing principal neurons, in for example the hippocampus 
||3^], the thalamus and the locust olfactory system [^. The 
mechanism by which synchronized oscillations are generated 
in the brain is only partly understood. In vivo many different 
rhythms of different frequency have been observed. Pharma- 
cological manipulations of slices in in vitro experiments have 
elucidated some aspects of the synchronization mechanism. 
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For instance, Whittington et al showed that GABA^- 
mediated inhibition is responsible for the synchronization in 
hippocampal sHces. SUce experiments, however, suffer from 
the drawback that the natural afferents are cut, and therefore 
the synaptic activity giving rise to the in vivo variability is 
absent. Recent theoretical work has shown that in fact syn- 
chronization by mutual inhibition is not robust against neu- 
ronal heterogeneities [|[| and synaptic noise [|]|. In theoretical 
investigations one usually only considers strong synchroniza- 
tion. In strong synchronization one imposes the strong con- 
straint that each neuron has to fire within a short interval from 
each other Here we propose that weak synchronization may 
in fact be more prevalent in networks connected by chemi- 
cal synapses. In weak synchronization the average neuronal 
activity is periodic, without each individual neuron having to 
fire at each period. Weak synchronization is for example con- 
sistent with the experiments in [|33|]. There are exceptions. 
For example, recent experiments on weakly electric fish show 
that neurons in the pacemaker nucleus are strongly synchro- 
nized There, however, synchronization can possibly be 
attributed to electric gap junctions. 

Weak synchronization is associated with a periodic drive. 
This drive is either generated externally, as is the case with 
Stochastic Resonance 35 1, or it is generated intrinsically 
by the network as it happens here. The neuron then skips 
periods, which it can either do deterministically (usually one 
peak in ISIH), or stochastically (multimodal ISIH). In both 
cases the network dynamics consists of clusters of neurons 
firing together But in the latter case the neuronal composition 
of the clusters varies with time. A neuron that at a certain 
point fired with a certain cluster A, can fire the next time with 
another cluster B. This mechanism yields a synchronization 
that is robust against noise, and neuronal heterogeneity. 

Oscillating neural assemblies do play an important role in 
the functioning of the invertebrate nervous system. In a se- 
ries of seminal experiments on the bee and locust olfactory 
system, Laurent et al. have shown that different odors acti- 



vate overlapping ensembles of projection neurons 136|. The 
periodic discharge of the ensembles is coherent on a cycle- 
by-cycle basis. Odors may be classified from the temporal fir- 
ing pattern of projection neurons '^%. Synchronization of the 
projection neurons may be abolished by applying picrotoxin, 
and without changing their individual response characteristics 
iQ] . This desynchronization was shown to impair the ability of 
bees to distinguish two closely related odors [p2|], and subse- 
quently a population of neurons was found that was sensitive 
to the synchronization of projection neurons [|8]]. The ability 
to distinguish between two odors, based on their spike trains, 
was then reduced under desynchronized conditions. 

We find that the information content, defined by the Shan- 
non entropy of the spike-time distribution, contains three con- 
tributions. First, the jitter in the spike times around the cluster 
firing time. Second, the distribution of the number of cycles 
between two consecutive spikes, and finally the correlation 
between consecutive ISIs. Our analysis has been performed 
on the (average) output of a single neuron. One has to await 
the development of fast computational techniques to tackle the 
more challenging problem of quantifying the information out- 



put of the network, while taking into account the correlation in 
spike times between different neurons due to the cluster state. 
Our single-neuron spike-train results, however, may have di- 
rect relevance to recent experimental work on striatal neurons 
ijS^]. Striatal neurons can be in the down (hyperpolarized, qui- 
escent), or in the up-state (depolarized, noisy). The transitions 
to the up-state are precisely timed and synchronous. The fine- 
structure in the spike train is asynchronous. Wilson et al. [ ^9| ] 
have suggested that the brain may use these two channels to 
encode different types of information. 

We have studied the dependence of these oscillations on 
network parameters. We find that there is difference between 
small (around ten neurons) and large networks (a few hun- 
dred neurons), under the conditions of having a fixed total 
synaptic drive per neuron. For small networks one needs 
more noise to drive the subthreshold network into stable os- 
cillations. These oscillations are very robust against increases 
in the noise level, and the fluctuations in the time between 
two cluster firings (cycle length) increases with the amount of 
noise. For large networks strong noise causes an instability, 
the stable cluster size for a given amount of noise becomes 
too small to inhibit out of sync neuronal discharges. For inter- 
mediate noise-strengths the neuronal dynamics self-organizes 
itself into a stochastically synchronized state. We also find 
that the farther the network is below threshold, more noise is 
necessary to induce a spiking state. The mechanism to create 
the oscillations is due to the competition between the excita- 
tory de-inactivating, and the inhibitory effect of the synaptic 
drive. Each cycle will de-inactivate neurons, until they are 
excitable again. The neuron then has to await the decay of in- 
hibition created by more excitable neurons. For some param- 
eter values the latter stage is absent, and the dynamics is fully 
deinactivation dominated. The important time-scales in the 
dynamics are the deinactivation time-scale ri and the synaptic 
decay time Ts- The cycle or population period scales directly 
with Ts- Our results therefore predict that by pharmacologi- 
cally decreasing one can increase the cycle frequency. 

In summary, the brain has circuitry capable synchronizing 
with heterogeneous components, and in the presence of noise. 
The spike trains of the synchronized discharge still contain in- 
formation. Whether the brain utilizes this mechanism to syn- 
chronize, and more importantly whether it uses the informa- 
tion in the precise temporal sequence is still an open question 
awaiting further study. 



V. ACKNOWLEDGEMENTS 

This work was partially funded by the Northeastern Univer- 
sity CIRCS fund, and the Sloan Center for Theoretical Neuro- 
biology (PT). We thank TJ Sejnowski for useful suggestions. 



8 



[1] Singer W and Gray CM. Visual feature integration and the tem- 
poral correlation hypothesis. Annu. Rev. NeuroscL, pages 555- 
586, 1995. 

[2] Whittington MA, Traub RD, and Jeffreys JGR. Synchronized 
oscillations in intemeuron networks driven by metabotropic 
glutamate receptor activation. Nature, 373:612-615, 1995. 

[3] Steriade M, Cormick DA, and Sejnowski TJ. Thalamocorti- 
cal oscillations in the sleeping and aroused brain. Science, 
262:679-685, 1993. 

[4] MacLeod K and Laurent G. Distinct mechanisms for synchro- 
nization and temporal patterning of odor-encoding neural as- 
semblies. Science, 274:976-979, 1996. 

[5] Wang XJ and Buzsaki G. Gamma oscillation by synaptic inhi- 
bition in a hippocampal intemeuronal network model. J. Neu- 
roscL, 16:6402-6413, 1996. 

[6] Tiesinga PHE, Rappel W-J, and Jose, JV. Synchronization in 
networks of noisy intemeurons. In Bower J, editor, Computa- 
tional Neuroscience, pages 555-559. Plenum Press, New York, 
1998. 

[7] Sillito AM, Jones HE, Gerstein GL, and West DC. Feature- 
linked synchronization of thalamic relay cell firing by feedback 
from the visual cortex. Nature, 369:479^82, 1994. 

[8] Jahnsen H and Llinas R. Ionic basis for the electroresponsive- 
ness and oscillatory properties of the guinea-pig thalamic neu- 
rones in vitro. J. Physiol, 349:227-247, 1984. 

[9] Wang X-J and Rinzel J. Spindle rhythmicity in the reticularis 
thalami nucleus: synchronization among mutually inhibitory 
neurons. Neuroscience, 53:899-904, 1993. 
[10] Wang X-J, Golomb D, and Rinzel J. Emergent spindle oscilla- 
tions and intermittent burst firing in a thalamic model: specific 
neuronal mechanisms. Proc Natl Acad Sci USA, 92:5577-5581, 
1995. 

[11] Golomb D and Rinzel J. Synchronization properties of spin- 
dle oscillations in a thalamic reticular nucleus model. J. Neuro- 
phys., 72:1109-1126, 1994. 

[12] von Krosigk M, Bal T, and McCormick DA. Cellular mecha- 
nisms of a synchronized oscillation in the thalamus. Science, 
261:361-364, 1993. 

[13] Bal T, von Krosigk M, and McCormick DA. Synaptic mecha- 
nisms underlying synchronized oscillations in the ferret lateral 
geniculate nucleus in vitro. / Physiol, 483:641-663, 1995. 

[14] Rinzel J, Terman D, Wang XJ, and Ermentrout B. Propagating 
Activity Patterns in Large-Scale Inhibitory Neuronal Networks. 
Science, 279:1351-1355, 1998. 

[15] Destexhe A and Sejnowski TJ. Synchronized oscillations in tha- 
lamic networks: insights from modeling studies. In Steriade M, 
Jones EG, and McCormick DA, editors. Thalamus. Elsevier, 
1996. 

[16] Destexhe A, Bal T, McCormick DA, and Sejnowski TJ. Ionic 
mechanisms underlying synchronized oscillations and propa- 
gating waves in a model of ferret thalamic slices. J. Neurophys., 
76:2049-2070, 1996. 

[17] Golomb D, Wang XJ, and Rinzel J. Propagation of spindle 
waves in a thalamic slice model. J. Neurophysiol, 75:750-769, 
1996. 



[18] Funke K, Nelle E, Li B , and Worgotter F. Corticofugal feedback 
improves the timing of retino-geniculate signal transmission. 
Neuroreport, 7:2130-2134, 1996. 

[19] Golomb D and Rinzel J. Clustering in globally coupled in- 
hibitory neurons. Physica D, 72:259-282, 1994. 

[20] Wiesenfeld K and Moss F. Stochastic resonance and the ben- 
efits of noise: from ice ages to crayfish and SQUIDs. Nature, 
373:33-36, 1995. 

[21] Longtin A. Autonomous stochastic resonance in bursting neu- 
rons. Phys. Rev. E, 55:868-876, 1997. 

[22] Stopfer M, Bhagavan S, Smith BH, and Laurent G. Impaired 
odor discrimination on desynchronization of odor-encoding 
neural assembUes. Nature, 390:70-74, 1997. 

[23] Riehle A, Grun S, Diesmann M, and Aertsen A. Spike synchro- 
nization and rate modulation differentially involved in motor 
cortical function. Science, 278:1950-1953, 1997. 

[24] Levin JE and Miller JP. Broeadband neural encoding in the 
cricket cereal sensory system enhanced by stochastic reso- 
nance. iVatore, 380:165-168, 1996. 

[25] Rieke F, Warland D, de Ruyter van Steveninck RR, and Bialek 
W. Spikes: exploring the neural code. MIT press, Cambridge, 
1997. 

[26] Softky WR and Koch C. The highly irregular firing of cortical 
cells is inconsistent with temporal integration of random EP- 
SPs. J. NeuroscL, 13:334-350, 1993. 

[27] Shadlen MN and Newsome WT. Noise, neural codes, and cor- 
tical organization. Curr. Opin. Neurobiol, 4:569-579, 1994. 

[28] Shadlen MN and Newsome WT. The variable discharge of cor- 
tical neurons: implications for connectivity, computation, and 
information coding. J. NeuroscL, 18:3870-3896, 1998. 

[29] Huguenard JR and McCormick DA. Simulation of the currents 
involved in rhythmic oscillations in thalamic relay neurons. /. 
Neurophys., 68:1373-1383, 1992. 

[30] McCormick DA and Huguenard JR. A model of the electro- 
physiological properties of thalamocortical relay neurons. J. 
Neurophys., 68:1384-1400, 1992. 

[31] Tiesinga PHE and Jos6, JV. Spiking Statistics in Noisy Hip- 
pocampal Intemeurons. Proceedings Computational Neuro- 
science 1998, 1999. 

[32] Rodieck RW, Kiang NY-S, and Gerstein GL. Some quantitative 
methods for the study of spontaneous activity of single neurons. 
Biophys. J., 2:351-368, 1962. 

[33] Traub RD, Whittington MA, Colling SB, Buzsaki G, and Jef- 
freys JGR. Analysis of gamma rhythms in the rat hippocampus 
in vitro and in vivo. J Physiol 493:471^84, 1996. 

[34] Moortgat KT, Keller CH, Bullock TH, and Sejnowski TJ. Sub- 
microsecond pacemaker precision is behaviorally modulated: 
the gymnotiform electromotor pathway. Proc. Natl. Acad. ScL, 
95:4684-4689, 1998. 

[35] Gluckman BJ, Netoff TI, Neel EJ, Ditto WL, Spano ML, and 
Schiff SJ. Stochastic resonance in a neuronal network from a 
mammalian brain. Phys. Rev. Lett., 77:4098^101, 1996. 

[36] Laurent G and Davidowitz H. Encoding of Olfactory infor- 
mation with oscillating neural assemblies. Science, 265:1872- 
1875, 1994. 

[37] Wehr M and Laurent G. Odor-encoding by temporal sequences 
of firing in neural assemblies. Nature, 384:162-165, 1996. 

[38] MacLeod K and Laurent G. Who reads temporal information 
contained across synchronized and oscillatory spike trains. Na- 
ture, 395:693-698, 1998. 



9 



[39] Stern EA, Jeager D, and Wilson CJ. Membrane potential of si- 
multaneously recorded striatal spiny neurons in vivo. Nature, 
394:475^78, 1998. 



10 




FIG. 1. (a) Phase-plane plots of the neuronal dynamics for the variables h and V . The plotted nuUclines correspond to dh/dt = 
(dot-dashed line), and dV/dt = (solid lines), I = —1.0 (top) and / = 0.0 (bottom). The phase trajectory of a neuron released after 10 ms 
(dotted), and 200 ms (dashed) with hyperpolarizing current pulse / = — 1. (b) Top curves for V, and bottom curves for h, plotted as a function 
of time for Ts = 5 ms < rc(solid lines), and curves Ts = 16 ms > Tc (dashed), with = 2.0. (c) Phase-diagram Ts vs. gs. Stable 
oscillations for Ts > Tc{gs) are shaded while the unstable ones Ts < Tc{gs) are colored white. 
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FIG. 2. Voltage traces for neuron one of m N = 1000 neuron network of all-to-all connected with (a) uniform initial conditions, (b) 
random initial conditions, (c) uniform initial conditions with noise (D = 0.02, without the transient of 100 ms), for three different values of 
Ts = 16, 13, 9 (from top to bottom). A voltage and time scale bar is shown in the lower left graph. 
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FIG. 3. Noise-driven single neuron dynamics with autosynaptic feedback, (a) Phase-diagram of neuronal behavior as function of noise 
strength D, for different values of Ts ■ No activity (solid line), spontaneous single spikes (dashed line), spontaneous spike-trains (flUed circles), 
(b) ISIH for (top) Ts = 15 and D = 0.76; (bottom) Ts — 10 and D = 2. (c) Representative voltage (top) and h (bottom) time traces for 
D = 0.26 (solid lines) and D = 0.76 (dashed Unes) with Ts = 15. 
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a D=0.0000 



b D=0.0024 




FIG. 4. Each panel consists of a gray-scale coded instantaneous h value distribution for consecutive cycles (top), and the time-averaged h 
distribution (over at least 500 cycles) (bottom). From left to right, top to bottom the noise- values are -D = 0, 0.024, 0.08, 0.8. In the upper left 
panel (a) the delta function peaks in the h-distribution have been broadened to enhance visibility. 
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FIG. 5. Four panels with the same D values given in Fig. q. from left to right, top to bottom the noise-values are D = 0, 0.024, 0.08, 0.8. 
We plot in each panel from top to bottom the V and h functions of neuron one, the population average Vsyn of the synaptic variables Si, and 
the instantaneous firing rate / as a function of time. 
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FIG. 6. Here we plot the average period (filled symbols, left hand scale), and the periodicity (open symbols, right hand scale) as a function 
of D, for Ts = 8, 10, 13, 16 (for N = 1000) and for different system sizes N = 10, 20, 50, 100, 200, and 1000. 
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FIG. 7. We plot the ISIH (+) and the corresponding SOG (continuous Une) for (a) D = 0.008 and (b) D = 0.8, both for N = 1000, and 
Ts = 16. We show the entropy per spike (c) as a function of the number of states n and the jitter sigma; (d) as a function of the amount of 
correlation between consecutive ISI and sigma. 
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APPENDIX A: MODEL EQUATIONS 



In this appendix we more specifically define the models studied in this paper. The dynamics of the voltage V and the kinetic 
variables h, s, and ni are given by the following equations, 



C'm~53~ — Ic'a II Isyn CmS,^ 



dV 
dt 

ds 

— = kfF{Vpre){l - s) - s/Tsyn, 

m = moc{V). 

Here the currents are 

Il=9i{V-El), 

ICa = gCamoo{V)h{V - £'ca), 

The asymptotic values of the kinetic variables, and the h time-scale, are specified by 

moc = l/[l + cxp-(y + 40)/7.4], 

- l/[l + cxp(F + 70)/4)], 

Th = Hro + + exp{V + 50)/3]). 

(Al) 

The synaptic activity is implemented in a standard way [^]. is chosen such that a presynaptic depolarization higher than 
—35 mV will open the synaptic channels. 

F{V) = !/[! + exp~{V + 35) /2]. (A2) 
The total synaptic coupling in our all-to-all network is defined as 

Stot = X! ^^^^ 

i 

and is the same for each neuron. 

The standard physiological set of parameters we use in our calculations have the conductances ql = 0.4, gca = 1-5, and 
g^yj^ = 2.0 (in mS/cm^), the reversal potentials El = —70, Eca = 90, E^yn — —85 (in mV), and Cm — l^F/cm^, tq = 30, 
Ti = 500 (in ms), kf = 0.5ms^^, and (p = 1.3. 

APPENDIX B: SHANNON ENTROPY CALCULATION 

As mentioned in the main body of the paper, we quantify the information content in the ISI using the Shannon entropy, 

s = -J v{{t-})v{{tn) iog2 p{{tn), (Bi) 

for the distribution of the spiking times. In this expression tf is the ith spike time of the nth neuron, and V is the sum over all 
spiking time possibilities within a [0, T] interval. We do not try to evaluate this quantity for the whole network, since at present 
it is a very hard calculation to do. Here we will instead calculate the one neuron entropy in the network, which is expressed in 
terms of the distribution P{{Tn}) of interspike intervals t„, as 

S = -J P({r„}M{r„})log2p({r„}) 

--{N)J drPisiH (r) log2 Pisih (t) , (B2) 
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with (A^) the average number of events during time interval T. We also assume that consecutive ISIs are randomly independent, 

PiTi,--,TN) ^linPlSiniTn)- (B3) 

As explained in the main body of the text the ISIH obtained from our simulations is, to a good approximation, a sum of Gaussians 
(SOG), i.e.. 



Pisih{t) = y^c,G'(r|^i,o-, 



(B4) 



where 



1 



:exp[ 



(B5) 



with average /i^, standard deviation (Ti, and Ci is the relative weight for the i-th Gaussian contribution. From the calculated 
ISIH, we estimate these parameters from the weight c,, the average fii, and the standard deviation ct, of the ith peak. When 
(/ii+i — /ii) ^ (Ti, for all i, Eq. (B2) reduces to 



i i 

= 5'i + 5*2. 



(B6) 



These are the two contributions to the entropy of a multimodal ISIH. A contribution 5*1 due to the jitter around the average 
ISI value of a given state i, and the contribution 5*2 due to the discrete probability distribution of the number of cycles between 
two spikes. Note that Si depends on the accuracy with which the ISI can be recorded and detected (taken to be 1 ms here). In 
the other limit {jii+i — fii) < ai, we numerically evaluate the entropy from Eq. (B2) using the measured (binned) ISIH. 

In Eq. ( p2[ ) we have assumed that consecutive ISIs are independent. If consecutive ISIs are correlated, however, one can use 
the theory of Markov chains to evaluate the Shannon entropy of the spike trains. The ISIs can belong to n different states, and 
they have an equilibrium probability P^q = (ci, ■■, c„) to be in any particular state. From a given state i the ISI can jump to 
a new state j with probability Tji, which we can obtain from a return map. In the return map we plot the next ISI versus the 
current ISI, and then divide it into a two-dimensional set of bins bji. The bins are centered on multiples of the cycle length, and 
their width is also equal to the cycle length, then 



(B7) 



and when the ISIs are independent, Tji = cj (i.e. the next state does not depend on the previous state). The actual ISI in each 
state also displays some jitter, and it is distributed according to some ^^(t). We will assume that ipi is Gaussian (as before), and 
the entropy is then given by: 

^ -[{m+ 1)1 ■ [To) -Peq+Y^ C» l°gC' + ■ (rlogT) • Peq] 



s 



log 2^ 

S'l + ^2 + S'3. 



(B8) 



We have used the following definitions 
implied), ai 



1 is the number of ISIs, {Ta)ij = Tijai {TlogT)ij = T!y logTy (no summations 



/ dT(t)i (r) log (pi (r), and ■ denotes matrix multiplication. Here we give the explicit formula's for n = 2 

(Ta) 



(TlogT) 



aip 0:2(1 -g) 
Q!i(l-p) a2q 



plogp (1 -g)log(l -(?) 

(1 -p)log(l -p) qlogq 



(B9) 



(BIO) 



When the ISIs are independent then S'3 = 0, and Si and 6*2 reduce to their expression given in Eq. (| 
We have evaluated expression Eq. (B6) (see Fig. ^c)) using 

27^(±^ + i) 



CL + l±i = 7^ I - 

y-j = + (i 

(To = cr. 



27r(±i - i) 



L)Am, 



(BID 

Here 71 = 2L + 1, i = 1, ••, L; j = 1, ••, n; and A/i is the cycle length. 

Correlation in consecutive ISIs can be parameterized using a 5, with q— 1 + 5 — p'm Eqs. (B9) and (BIC). The explicit result 
for m — 100 is plotted in Fig. 0(d). 
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